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(54) SPEECH IMAGE RECOGNITION AND TRANSLATION DEVICE 

(57)Abstract: 

PROBLEM TO BE SOLVED: To improve the precision of translation by 
judging a phrase of a sentence consisting of an inputted continuous 
speech when features of an image of the whole mouth and its periphery 
corresponding to features of a speech indicate the end of the phrase of 
the sentence. 

SOLUTION: The device consists of a microphone 101, A/D converting 
ICs 102 and 107, a speech input part 103, a memories 104 and 109, a 
speech recognizing process part 105, a camera 106, an image input part 
108, an image recognizing process part 110, a translating process part 
111, a speech synthesizing process part 112, a D/A converting IC 113, 
an LCD 114, and a speaker 115. A phrase of a sentence consisting of a 
continuous input speech is recognized and the words constituting the 
sentence are recognized to perform speech recognition and translation. 
Here, when features of the image of the mouth corresponding to the 
speech indicate the end of the phrase of the sentence, it is judged that 
it is the phrase of the sentence consisting of the inputted continuous 
speech, the character and character string in the ending of this phrase 
are recognized, and a mark indicating the phrase is added to accurately 
recognize the phrase and word ending of the speech input sentence. 
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Jf^SMWi:, &«9i£A,ft BittS^— ^«rfB«Uf 

U T fl K L- ft i ^ H * X 3£ B R -T 5 fa R Hi S W- £ £ fit 
Lft:®«<z>#»o 2 otf>ffiKJ«l«J&>fe , A2? *ftft — X 

UftL*-&{-l2, A*£ixft— a^O^^A^^51^3C^3C 
**lir{fe«tt«lRiS1K. 

W &c*r j£ u ft □ ±# t -t <z>» £ >£« ^ i® & '7>4#t& a* s: o 
KKSRSS1K. 

L ft ^W^feeflFfciig-ffcU Mi- 

t***rV (1) ffWMSiiSiBt^ 
myiLtf**}) (2) t^6t«t5i*t-. Bi*?aAr 

ir-^Sr&S&fc^^r T J (3) W^^7v*^na<t 
M&ir-^^&S^t^^y (4) £*»fc«j£U 

^r — fn&V&fr&ffitkV, £ft> ^FcoA^&TL 
ft £ ¥U«r L,ft > £ tcte^-r-* SLPlEffex — 

[Ht3fc3S5] R«94 id*5V^r, *FJ&*A*£ixft:i*'J 



£t L iz £2 — 7=—^ 5 a**W 

L^SiCOl ^X . 

(1) ^Kffr- ^iC^LT , 3#f^T i IC^*?^^^ 

(D icistt l^w^t^-^ t/^v (2) (ctatgu/i 
^St^^-^^^-f s^^i', y^u (3) i^tai© 

U/LiBtti'T 5 — ^ ^ ^ ^ y (4) icfe^L-fcM^^-^i: 
BKfiRSSS. 

[If j^lg 6] |»3R3lliC*3V^T, Htc ffFBRiVi 
ft Btt S *x ft: Ml f ^ X ^ 60 Bfttl^ m \z. ^Lt8«ft * <0 
IgiE ^ ft: fi rf IE % 'fT p *«F E«8tR it ft w <h 5: » 

» ^ -r ^ * w n it sssfe a R g Be 

[0 0 0 1] 
[0 0 0 2 ] 

<7)ii: j; -6 ^ ^ ^^^-v-a >o^i*J^»t5ft 

iz^^x^m^fsjc^m^R ir^it^^r^ ^ o 

■50 Tl^o L/:^ot, rcO^^T*^. ^r^btlft^M^C 
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10 0 0 3 ] *#PI^5-3 5 7 7 6 (£ 

*; rafga»a^«itettaK»«j ) 

[0 0 04 1 11 014, :^)^9^^fF^f 
o i w^^-r^c ^ s-e*>s 6 WlBlfc:*3^"C. 100 
1I4*]SP«* 1 0 0 2lifFEB«I9aUi. 100 3 
ttSFBSfcB, 1 0 0 4!^/tH5, 1 0 0 5lifF*S 
ff. l 0 0 6i^fif-^ffi^^[J^-K. 100 7 
14#FIS«»»», I 0 0 8tt^tf-^. I 0 0 9!iv 

-r^, loioi^t-^r^, i o i i ttWf^fS* 

[0 0 0 5 ] 100111 -r-f * c^n-ir j/-^^ 

[0 0 0 6 ] ^W\Z?fi%) K> fcb LSfl 1 0 0 214, ? 1 

0 0 9 A^A^^n^^^^r^^^ A'ft-i-iClg^LT^) 
"9 ffi U ^FBKSB 1 0 0 3 }Ci££ 0 

[0 0 0 7 ] #F?BS&H5 1 0 0 3 14, =*— ^— KSfctt 
;W y^lcj: 19 A*$n7t»fFfi#l 0 1 1 
Ml HP OS 1 0 0 1 (7ym^lZ X "9 , -7Y^ 1 0 0 9^bA* 
Itt^Efl^ 9 tU LStf 1 0 0 2 £r*£T^ 0 S L/LWF 

0 7 icftjW*ixfc«2|li*^^^ — >-^tt:«LT:*WlS» 

[0 0 0 8 ] ^J^^FfzM 1 0 0 5ft w«ri^^Fi 100 

3ia 9tt»Lfc*FfcL#ffiUfcaRi§*«Ri§^— * 
itt. t'-^ryyi o i o&L*-* tf— # 1008 

[0 0 0 9 ] 1 0 0 4 14, ilPv^Oft^f^ 

[0 0 10] HRffi-T — *ffl*^e y # — Kl 0 0 6 14, 
ROM, 7 9 ;y y , K^-f * 9 ^^"bft 

1 0 0 6*>(bl4. BRfi^Mtebfc** 7^ K 
^W/Ot-^1 0 0 4i:f/7t5 o ^Lt, CO 
BRi§v*— #m*=£ V X — K1006 ^fdl^KlicD-b^) 

[ 0 0 1 1 ] #J*3£Hfci¥WW 1 0 0 7 (4, ROM, RA 
[0 0 12] tFM^BBtifcoT, Hft-ffi 
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SFSRJSaJw t * T 14 , « fW ±-4" 6^-1' R» 

10 t L T ift L T is * , * coig- O^t « IC # r ft£? 
BX*:»iRLr«^S*^:t*IIBttiCl4S*3t&i\ 
-tft;b^, 3 # g M ^ 3? # £ * E s jo a* p - s - 75 ft \ z M 

^®lcioi^T!4, g»^f^/:i^ ttBRtSJ: 19 

W^I5T-fc5. -enSrHm-r Sfc*Mcf4, 5?ftLft 

[0014] «*«»$fcfcif-eS^*n5£^G>J: 5 
ft, H:«ttH^3CS*-fe*5i«tt*l»«:R«L.rHW-f 

ft 5 c i:^r-^ftt\ ^^ooA^^^Simte^ 

»ff^ft^T*h>^ix5^f§o c t 9 ft, ttfe*)^^3t^ 

ni4ftbftv^ 
[0 0 1 5] tlf, WW:. ^LfbSSb 

L t ^ti SLO*«S*FB«ttBR *jm \z ~t -5 St ^M^^s^ 

5 - t -e#B*^«r»« LTIE L < «t5: > ^T- 
[0016] 

/'^{c, ^«^icft5^FBsSaR^S!4. ^» A: Hjf^ 

«- x o© ic u r sift <rc w« & « as L . ® * Bsa 
•^aR-rsBRttaast ^^a^ic^ft^^^ 
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#tt iwSE-f t; u r ^ < il^lc &> i^t , r. o^f ^>4#% \z.3& 

JbtttlcSEfc l r v ^ < a«^*5 ^ r , r co #F ^)«* M** 

JCS b □ t * Offl 53 iSfl? ««« 36* 3: « 3Cfl5 CO 

[0 0 19] SD^fiW^-^^E* L-T*5 

(1) ^^^^nsfcis*^--^*:*^^^ 

pt^-ij (2) t^f5«P)tt5i:*ic, &9i£A,f£ll!&x 
ixfc £ 49JB; Lit i: # ic^F^— * SViif-i? ^» G 

[0 0 2 0 ] *F3ft*A**ix&^*l»L,^i:*ic 
jfcuquffc-r — ^ ^> ^ 0 V-^&ffiih-t' % 

opt, ¥F<oW«ii«*?ixSfi:K^»3E*^fcJ5»l 

jo<^^-'J (1) (DKtf-^l^tr, »BTii:» 

^^'tf-^^MP i iom<o®m<DmmT 

i -licit l5^fc'tT"^^iSP i-lt^AP i o 

^x&n^-r-ffrbm^j^v (2) K»#&t?£* 
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(3) l^EiftLfc.Btt^--*' - * ^ '> (4) MI5^L-7l 

[0 0 2 1 ] 551^. *^»9Ht^ir*»»*^* 

f(? mi^^ * ^'^^ » i- *f ■ T^sa^ ^ ^0 IE * - ft 

[0 0 2 2 ] ^fc, fflIE®«^#«!i. ®*-r — 
[0 0 2 3 ] fflE^^fclE««:*Oi&t*^l*«r* 

[0 0 2 4] Jb^co «t o ttfllja<o*FB»BKft**^ 

E Pv & 1 1 i w "f S St Si "i 1 B 5^ B R » * 5 * ^ i ~" * • 
[0 0 2 5 ] 

[JSStOHttw^llS] SIT, BffiSr^SH-teA^^' *:* 

[0026] aitt, ie«fl5fflt-«s*^i® 

lCf^^<oH)LSIt*«lSt5:i:i^ , ^o ^ 
^ 5. 

[0 0 2 7 ] 111 t*5l^r , 1 0 1 ^7A3j>/.: 

mm ft, mrfmft, ^^^ >a 

[0 0 2 8 ] 1 0 2li 1 6 If? hwrto^/r^^A 

(a/d) i , ffrE-^-i' H01 rt»7-f 

^^ry^ia i}*^**^^** 4 *^ ^^^^ 

FWt>/y >yJSS*, 1 2 kHz-Ct^'J 

[0 0 2 9 ] 1 o aiitf^^Dii^Bffc^ . A/ DS 
ft I C 1 0 2 X- LJ > y£Zltz 1 6 h (7)^^x 

[0030] 10 4H miewFfco&^aci o 3t^t 
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[0 0 3 1 ] 105 tt#^rB«*&aSB-Cfc 0 . ^ e Li 1 

yj^?, ^-p-9}%7, ¥j»eiB«k£s, mi*, 

^rai o 5f*h-*5^t y Lfeaa it 

y T-£>£ a :intt. P'Jrti*. CCD^^7t^!), 

U:^;bi*:TR»F£«fei"£Aff>D<75»£ SrMtt'T— * 

[0 0 3 3 ] 107iil6t> h (7>T ^-n ? / =f is * A- 
(A/D) SilCT'^^ ffllECCD^^ 7 1 0 6£> 

IT, 12kHz X^iyf V y^ltry^b 

[0 0 3 4 ] 1 0 SttMft&tJiA^SB^fc!? , fttJfcA/ 

DS:m i c i o 7T*fyyy V^L/c 1 6 f h« 
^l:^lt, v-y Tz^r — *? \s/i-"t — '?\Z 

v-y w^s^ot ja»*A 

[0035] 10 9 H "3 108ia^ 
■?&A,&TH*r--^ #<Jr> fl ^|g#Ft7)l 7 I X» 

oajKurK-r — ^SrffitgLr*5< iz#><D* * y t-&> 9 . 

gi^o>^yT^5 tt aatwflfc^— ^<7>p<^ y cos 
cpu^y7h^7feir''fTorbiS 

cO/n— K <? =c-TX'ft<z>Xh&\< s « 
[0 0 3 6 ] 110 JiBj«ftfgS6*a3Sffl5T-*> "9 > BfJlE* ^ 

y i o 9 {z^z &£ftt-mmw&'r—* \zm L,x . t\s 

^^7^/^, Pf«S«, 2«Mfc»« f 4#t& 
SCft-r — * J*, ffiJ«^®*a3ai5 1 0 5 ft tCSS^T. 40 

!)«:S»itMir^< 0 Vftowtriatt. cpu 
[oo37] i i flJjiEffF»«toa« i o 5 a> <b 
&m, mm, fern, giH*^6si^ii:9or*^h 



[oo38] i i 2»i. ffiEfltst&ssm i i i a*«bHi* 

£ 41 a M & £ . SBXWB L;fc*FicXft L T-^-- 



[0 0 3 9] 1 13lil6lfyh toy* iff )V / T -f u # 
(D/A) SHlGt*>9, fflSE^^-^-SteaSS 1 12 

^^.f/t'^ (LPF) MSLT^MtKl2k 

h z -er-T-ss ^ft^-i^^^^c 
[0 0 4 0] 114J1 ^Ff^^l¥^&^i&i£^8^ 

CD) T'&>5o 

[0 0 4 1 ] 1 1 5H ^WfZMm^*: 

[0 0 4 2 ] ht5fc*wr 

K 6 ^ffi cr> - #i| $r R ^ ^ 6 & tf> f fo 5 c 
[0 0 4 3 ] [Hi Lfc.^^ 7106, 1?-Jrtl^, C C 

[0044] 2 0 111 W^^ilt^^^ 
ffiSOB**^(0^iS«F^«*LT^6A^»Jl:®tt-e 
ct^^c B«Oj|?«ftrtL. ffiim F c h x« n K y h 

r. Hta. fmy^y >ym&&& 1 2 kH zic 
y 2 4 0 > h ;c L/t^^n 

2 0ms U- — AmfiCT-^F7N.*lwl^»l*-e:r*0i^ 

— A^4^rll!fifeofe»3i^^tCj®ffl-r-5i: . 2 0 m s T- 1 
7 \s — J*<DW&$:?Vi } 0 i&tS Z. blZte&<DX\ 1 5 

ONTSCf«tti»t?*)5 3 0 7 "9 tB? 

[0045] 2 0 2(1 Wi*) ihA,1£W1& 2 0 1 M*t L 

[0046] 203 ^. li» 2 0 2 M** LT. iE^fl:, 
a C<7>iij^ 2 0 3W»»a«. Hjfife^ 
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[0 0 4 7 ] B3«t, &F&X*Wte&&*)jLt?1ti>><nZ 
M 0 1 7 1 0 6 LfctOTfcS, » 

fc* 0 2^#.BSLrRflLfcJ: PC0»#(7)iIf»Sg 

[0 0 4 8 ] 3 0 1 it, WWauMBttSTftG >i£i?/L«>tf> 

60 3 0 3iii/yx^»), fF^4t6Aft©«« 

[0 0 4 9 ] 3 0 oit^WRLmfe^—? Zfe&'fZtz 

My-^r^o. *Fft #y-^^3 0 6 &pittft 

SHBIE^S&^-Cii , ^it^r-tr — -?^ 3 0 6 liA/D 
^» I c 1 0 2 M*i*L, Httft-S-^ — ^ 3 0 7 fi A 
/D3E* 1 c 1 0 7 £*BH-5. 
loooo] I4!l 5 

[0051] A* ^ixfc^K^FM j: S^fijCW^rtS 
[ 0 0 5 2 ] 40 1li, A* £ix£:^fS^FiC£ ££f§ 

3E0i)ic*5tf£ r=r^^v^ K=r-x^^j 

[0053] 4 0 2-4 0 4!! *F*>»flft Zfeft-fh 

tz£><D7 u — AfetF^fy^!) f^,** 2 4 0#>f 

V hie Lfc^^(7> 2 0ms Mwf^#i»7 U — 2±X"fc> 
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[0054] fc#F^^a*ffiWA*o*FlB*E*: 
■ff 5 - t te^fsr-C £>© ii. ¥Ftf>fe* S (BR) A^fc*i--> 

13 9xy/\j & fa^x u~ & o &. l ; „ r = 7" 

^r)xy/%j # ra 7^>'^; Mfro*:*;, r - 

«**«>5^«:£ LT. *F<D*&* 9 (fill) 

(RJ&) ^*5iJ 5PO»*^#a Ufcc 4 0 5-407 
te, f ^#IS7 4 0 2 - 4 0 4 t'^ I tfF^^ 

£0 attics 7 L'— ^ & y 

£24 Otfi'^ hfCLfc^lErZ) 2 0 m s MftO^F^Ma 

M4 0 7T?*$ii5#«tt, r F^f^^j ooi5^7^> 

[0055] 4 0 8-4 1 211 7^* ^ nfc^JS*FJ- 
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ti^K-. r.^^j 4 0 8 > rfij 4ioi^i;^ 

£>j 4 1 2i:(7)ffl!:^7^v'^4 1 l^r?i,\t5c 

[ooo6] 4 i 3i3, ssksa r&m/n/zzx-r 

^ai^lT^^ Twh ereis the pa 
rk?j Irr^^hT'^t^, ^rlt. ^. 
4 1 3 it. ^jz^stisX^wmfj-tz. 

[0 0 5 7 ] 1511 Z<D& 5 ft*fSWI-ft5*K!B* 
[0058] ( a ) fl *S#*<IH»i-*Ffc L- 

r^^Lf- rajaj o^Fco^j^r-^^o Ei5(7> (b) 

10 0 5 9] 15 ( a ) Cfc'Pt, !2lGD$t$|fc;*|S] (BK 
[0 0 6 0] ERB 501 ^"TS^il 

S»t*Lr^5, «Fa*«££ixTi^fc^«#ll S 

l fcm**Jffi*36iatoix5 (t ^r.r*, P'Jxfl 3Mftft(c 

j#f*-*Sr^y i:iIEftLT*5#, ^Cf^Lt^T-^ 

^y S**r»o**-y«:%:t5 0 «.t 
il 0. l^Saco^F^-^^ft^-C^-S^fi^rfip^ 

[0 0 6 1 ] 05 ( b ) rC&tf (1) 506 

H 5S«SW«c«iM*ix-5^6r>afl0Jc*5i>r, *> 5 as BBSS 
r^< ^^jc^s*yj^K(7D^* y gfin V X' 

foZ>* 5 0 7(i, (1) 5 0 6C^(j67^' h7 

K^WAT'fcS. 5 0 811 ^^V (1) 5 0 6M& 

7 Y h 7 K U^WA 5 0 7 (7)/7tr K l^^iCtPf 
-^^7-Y h t, 7^F7K^\VA5 0 7 4:>fy^!j 

^> htr^< 6 roii^^^^y (i) 5 0 6co^si 
Lfc*fetf«l«feSJ[7' KU'^l:RotB«l:aifc»5 

i£-T 0 tf'JxJl (15 (a) (OKB5 0 1l:*5^T, Si^ 

15 (b) Ci&ttS;*^ 
y (i) 5 0 6o««u:«i&iii5. B5 

(a) CDERB5 0 1 Mfcl^T. fc*?***^* 
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5E£c2 *>*430>*^S?P^— ?»1 Hi 5 ib) Mrfci": 
5*~y (l) <7>«*2!c«£i£SH£. (Kr^ 5 0ii: 
*H '•-<:, E*2*0-r— 1 £ »; t,i|tj:^ 

^hTt^^ii. * ~ \) (l) 5 0 6 M $5 1- x T . gJ^2 

r*^5. i^J^t, f^?^t6ri^;:i^ , ebb 5 o 

2 tw*-TS?S?v*— * & X fj ic , Q(% I itf^— * 1* ggiSc 2 

tr-^rf !:^^tri fctf'sjffircfcfi. y 
/(? (l) 5 0 6©*^-* !»- HicHLr{i»"P»-<5« 
[0 0 6 2 ] b. #^**SiS«*£itf> 

-^lafFt^^i^j^L , s»j y *i«*n-rs r. t 

^^i5c 15 (a) iC*5it5EBB 5 0 3365*^^— * 

[0 0 6 3] 15 (b) ICrJotfS^^r y (2) 50 9 

$ ixtz.fi: h il &t5> «b ft 1 fciitSflfT'— * lifFTl) 

fc5o 5 1 Oil .^^-y (2) 5 0 9iCj3ij^7^' h7 
KU^WAffcS. 5 1111 ^ (2) 5 0 9i:fc 
<7 S y - K r K ^ R AT- So ^ U S*W . Mi 

ftW-rS^id^-rS^SaicfeWAi-frS, .^D (2) ic 
5 3 <nm**^\t. mo (a) C*i(5i7 5Srfl5 0 3« 

IS, J^V (2) 5 0 9 ^mWPr*— * w*#ii^Srq3 
D, B5 (a) iC*tKM5 0 2. 5 0 

SJP'r r — i , 2 , 3 ii , Si 5 ( b ) ic.^-*- ^ -*e y 
(D 5 0 6KM^ J J (2) 5 0 9i:i^^^ii ( c 
n^^^^-y (i) Sit* (2) ic^^iA^ix^^FaET^ 

V r -^!t. 5 1 2 JC^i"^ 9 ^^^^lUC^^tb-r^. t ^ 
[0 0 6 4] 15 ( b ) SCiJl} 6 ^ * y ( 3 ) 513 

fa, (i) 5 0 6 tn«i:. s«en^*ffj$ns 
^icMft/hPi^^^ 1 ; Sflm*:»o^ * y r-^^o 5 

1411 /^V (3) 5 1 3lwfttf<57'r h r K^W 
AT^5 e 5 1511 ^^y (3) 5l3tC*3i^iy — 
K7K^RATfc5. &*nS£, h 

T Ku^WAo l 4 (7)^f7 Ku^!CHiteT f — ^4r9-f 
rU 7^ h7 Kt/^WA 5 14 & i' V ^ !l ^ > h LT 
r.^»f^«r^ ^ y (3) 5 I 3<D9cmT KU^*» 

£0 fl H5 (a) ^EP^SOlf^^r, M^&OEig 
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5 (b) ^ ] J (3) 5 1 3(7)««l!:f*ii 

[ 0 0 6 5 ] H 5 (a) CD KM 5 0 1 iC*3^T , 

*i::*fri;ufc.nff>fc# om*^--*^ is (b) k*s 
tfs^^-y (3) o i 3^nn*2ici : #3A*nri / ^ e 

KM 5 0 1 |C*3^T, E*S2W7 f — *3&*E«1 ^t*— * 
infli^^^ixr^^Wlj:, (3) 5 13M 
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